Skip to content

Support SST Decoder#284

Merged
marsishandsome merged 2 commits intotikv:masterfrom
marsishandsome:feature/support-sst-decoder
Oct 14, 2021
Merged

Support SST Decoder#284
marsishandsome merged 2 commits intotikv:masterfrom
marsishandsome:feature/support-sst-decoder

Conversation

@marsishandsome
Copy link
Copy Markdown
Collaborator

@marsishandsome marsishandsome commented Oct 9, 2021 •
edited
Loading

Signed-off-by: marsishandsome marsishandsome@gmail.com

What problem does this PR solve?

close #285

support decode SST to Key-Value

subtask of tikv/tikv#11007

What is changed and how it works?

RocksDB provides a java API SstFileReader to decode SST to Key-Value pairs. We can wrapper this API in client-java.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to be included in the release note

@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch 6 times, most recently from 4bcdc15 to 6cb7646 Compare October 11, 2021 02:50
@marsishandsome
Support SST Decoder
e0c00f6 
Signed-off-by: marsishandsome <marsishandsome@gmail.com>
@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch from 6cb7646 to e0c00f6 Compare October 11, 2021 02:57
@marsishandsome
Copy link
Copy Markdown
Collaborator Author

marsishandsome commented Oct 11, 2021

/run-all-tests

@marsishandsome marsishandsome mentioned this pull request Oct 11, 2021
Merged
Little-Wallace
Little-Wallace approved these changes Oct 13, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@Little-Wallace Little-Wallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andylokandy
Copy link
Copy Markdown
Contributor

andylokandy commented Oct 13, 2021 •
edited
Loading

I suggest checking SSTMeta.data_encode. This should help avoid silent bugs after enabling API V2.

@marsishandsome
Copy link
Copy Markdown
Collaborator Author

marsishandsome commented Oct 14, 2021 •
edited
Loading

I suggest checking SSTMeta.data_encode. This should help avoid silent bugs after enabling API V2.

Cool. Can we do this job as a subtask of API V2?
We can update to API to pass the SST Metadata file and leave the implementation empty.

@marsishandsome
Copy link
Copy Markdown
Collaborator Author

marsishandsome commented Oct 14, 2021

/run-all-tests

@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch from d0dd3c8 to a6f13ea Compare October 14, 2021 08:44
@marsishandsome
Copy link
Copy Markdown
Collaborator Author

marsishandsome commented Oct 14, 2021

/run-all-tests

@marsishandsome
add backupmeta decoder
5869fb8 
Signed-off-by: marsishandsome <marsishandsome@gmail.com>
@marsishandsome marsishandsome force-pushed the feature/support-sst-decoder branch from a6f13ea to 5869fb8 Compare October 14, 2021 09:01
@marsishandsome
Copy link
Copy Markdown
Collaborator Author

marsishandsome commented Oct 14, 2021

/run-all-tests

@marsishandsome marsishandsome merged commit bbe4f9b into tikv:master Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andylokandy andylokandy andylokandy approved these changes

@Little-Wallace Little-Wallace Little-Wallace approved these changes

@ti-srebot ti-srebot ti-srebot approved these changes

Assignees

No one assigned

Labels

status/LGT2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

support decode SST to Key-Value

4 participants

@marsishandsome @andylokandy @Little-Wallace @ti-srebot